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I  Introduction 

This  is  a  continuation  of  one  of  the  previous  studies,  which 
w»s  published  as  Office  of  Naval  Research  Report  80-4,  (Samejima, 

RR-80-4),  under  the  same  title.  In  the  previous  report,  two  subtests 

of  the  original  Old  Test,  i.e..  Subtest  1  and  Subtest  2,  were  used, 

separately,  in  place  of  the  Old  Test,  and  the  estimation  of  the 
operating  characteristics  of  the  discrete  responses  was  experimented 
upon  each  of  the  two  subtests.  The  main  features  of  this  new  method 
are:  1)  the  number  of  test  items  used  as  the  basis  for  the  estimation 
is  less  than  that  of  the  original  Old  Test,  i.e.,  twenty-five  in  each 
subtest  agair.st  thirty-five  of  the  original  Old  Test*  and,  consequently, 
the  amount  of  test  information  is  less  than  that  of  the  original  Old 
Test;  2)  unlike  the  original  Old  Test,  the  test  information  function  of 

each  subtest  is  not  constant  for  the  interval  of  ability  of  our 

Interest,  and,  therefore,  we  need  the  transformed  ability  in  addition 
to  the  original  ability  dimension,  so  that  the  resultant  test 
information  for  the  new  ability  scale  be  constant;  and  3)  in  doing, 
the  method  of  moments  for  fitting  polynomials,  which  turned  out  to  be 
the  least  squares  solution (Same j ima  and  Livingston,  RR-79-2), 
is  effectively  adopted.  Out  of  many  combinations  of  a  method  and  an 
approach  for  estimating  the  operating  characteristics  of  the  discrete 
responses  (Samejima,  1977,  RR-77-1,  RR-78-1,  RR-78-2,  RR-78-3,  RR-78-4, 
RR-78-5,  RR-78-6),  the  combination  of  the  Simple  Sum  Procedure  of  the 
Conditional  P.D.F.  Approach  and  the  Normal  Approach  Method  was  selected  for 
the  experimentation.  We  use  the  same  group  of  five  hundred  hypothetical 
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examinees,  whose  ability  levels  are  one  hundred  equally  spaced 
positions  starting  from  -2.475  and  ending  with  2.475  on  the 
ability  dimension  with  five  examinees  placed  at  each  position; 
thus  they  represent  the  uniform  distribution  of  ability  for  the 
interval,  (-2.5,  2.5)  . 

In  the  present  study,  the  third  subtest.  Subtest  3,  is  used 
in  place  of  the  original  Old  Test.  The  number  of  test  items  is  even 
less  than  those  of  Subtests  1  and  2,  i.e.,  fifteen  aginst  twenty-five. 
Another  big  difference  is  that  for  Subtest  3  the  amount  of  test 
information  is  much  smaller  around  the  two  endpoints  of  the  ability 
interval,  (-2.5,  2.5)  ,  and,  consequently,  the  maximum  likelihood 
estimate  of  ability  turned  out  to  be  either  negative  or  positive 
infinity  for  some  hypothetical  examinees.  For  this  reason,  some 
adjustment  had  to  be  made,  and  we  chose  to  use  a  modified  maximum 
likelihood  estimate,  which  was  introduced  in  a  previous  study 
(Samejlma,  RR-81-1) . 
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II  Rationale  behind  the  Modified  Maximum  Likelihood  Estimate  t* 
Let  0  be  ability,  or  latent  trait,  which  assumes  any  real 
number,  such  that 


(2.1) 


_0D  <  0  <  CD 


Let  g  (*l,2,...,n)  denote  an  item,  and  x  (*0,1,2, ...  ,m  )  be  a 

8  8 

graded  item  response  to  item  g  .  The  operating  characteristic, 

P  (0)  ,  of  the  graded  item  response,  or  item  score,  x  is  defined 

*g  E 

as  the  conditional  probability,  given  ability  6  ,  with  which  the 


examinee  obtains  the  item  score  x  for  item  g  .  In  the  normal  ogive 

g 

model,  this  operative  characteristic  is  defined  by 


(2.2)  Px  (0)  -  (2tt) 

8 


-1/2 


/a  (6-b  ) 

a  (e-b  .  i ) 


n 


du 


8  V1 


where  a  (>  0)  is  the  item  discrimination  parameter  and  b  is 
K  X 

g 

the  item  response  difficulty  parameter  which  satisfies 


(2.3) 


“  bo  <  bl  <  b2  < 


^  b  ^  b  -  * ,  \ 

m  (m  +1 ) 

g  g 


Table  2-1  presents  the  item  discrimination  parameter,  a  , 

D 

and  the  item  response  difficulty  parameters,  b  ,  for  x  *  1  and 

Xg  8 

x  ■  2  ,  for  each  of  the  thirty-five  test  items  of  the  Old  Test. 

In  the  same  table,  also  presented  are  crosses  indicating  the 
items  included  in  each  of  the  three  subtests,  i.e.,  Subtests  1,  2 
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and  3  .  We  can  see  In  this  table  that  Subtest  3  la  a  subset  of 
Subtest  lt  as  vail  as  a  subset  of  the  original  Old  Test,  with  the 
exclusion  of  the  five  easiest  teat  itens  and  the  five  moat  difficult  items. 
Let  A  (6)  denote  the  basic  function  of  the  item  score  x  , 

X  ft 

8  s 
which  is  defined  by 

(2.4)  Ax  (0)  -  ^  log  P  (6)  . 

g  °  g 

The  item  response  information  function,  I  (0)  ,  for  the  item  score 

*8 

x  is  obtained  from  the  basic  function,  or  directly  from  the  operating 

O 

characteristic.  We  can  write 

(2.5)  Ix  (0)  -  -  Ax  (0) - *4"  lo*  px  (6)  * 

8  g  36  g 

The  item  information  function,  I  (0)  ,  is  defined  as  the  conditional 

g 

expectation  of  the  response  pattern  information  function,  given  6  , 
such  that 

m 

g 

(2.6)  1(0)  -  E[I  (8)|0]  -  Z  I  (0)  P  (0)  . 

8  g  xg-0  xg  xg 

Let  V  denote  the  response  pattern,  or  a  vector  of  n  item 
scores  such  that 

(2 .7)  V  ■  (x. ,  x0 ,  •«.,  x  ,  ...,  x  )  . 

By  the  assumption  of  local  independence  (Lord  and  Novick,  1968),  the 
operating  characteristic  of  the  response  pattern,  Py(0)  »  or  the 
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conditional  probalility,  given  ability  6  ,  with  which  the  examinee 
obtains  the  response  pattern  V  ,  is  the  simple  product  of  the  n 
operating  characteristics  of  the  graded  item  scores,  such  that 

(2.8)  PV(Q)  -  n  Pv  (6)  . 

Vv  8 

We  can  write  for  the  response  pattern  information  function,  1^(0)  • 
such  that 

32 

(2.9)  1__(0)  - - r  log  P„(0)  •  l  I  (6)  . 

36  XtV  Xg 

© 

The  test  information  function,  1(8),  is  defined  as  the  conditional 
expectation  of  the  response  pattern  information  function,  given  0  , 
such  that 

(2.10)  1(6)  -  l  1/0)  Pv(0)  . 

It  can  be  shown  that  the  test  information  function,  which  is  defined 
by  (2.10),  is  also  the  sum  of  the  n  item  information  functions,  so 
that  we  can  write 

n 

(2.11)  1(0)  -  I  I  (0)  . 

8-1  8 

The  rationale  behind  the  method  of  estimating  the  operating 
characteristics  of  discrete  item  responses  without  assuming  any 
mathematical  form,  using  "Old  Test"  with  a  known  set  of  item  response 
operating  characteristics,  which  has  a  non-constant  test  information 
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function,  has  been  described  (Samejima,  RR-80-2).  In  this  method, 
the  square  root  of  the  test  information  function  has  an  Important 
role.  This  fact,  together  with  the  findings  about  a  certain 
constancy  of  the  square  root  of  the  item  information  each  test  item 
can  provide  for  the  entire  range  of  ability,  regardless  of  its 
difficulty  and  discrimination  power  (Samejima,  RR-79-1),  suggests 
that  it  will  be  more  fruitful  to  observe  the  square  root  of  an 
information  function,  rather  than  the  information  function  itself, 
in  future  studies. 

Figure  2-1  presents  the  square  root  of  the  test  information 
function  of  Subtest  3  by  a  solid  curve,  in  comparison  with  that  of 
Subtest  1,  of  which  Subtest  3  is  a  subset,  which  is  drawn  by  a 
dashed  curve.  In  the  same  figure,  also  presented  is  a  horizontal 
xine  with  the  height  of  4.65  ,  which  indicates  the  square  root 
of  the  vest  information  function  of  the  original  Old  Test,  whose 
test  information  function  is  approximately  21.63  for  the  range 
of  ability  6  indicated  in  the  figure. 

We  can  see  in  this  figure  that  the  amounts  of  information 
these  three  tests  provide  us  with  are  approximately  the  same  around 
6  -  0.0  .  While  the  original  Old  Test  retains  a  constant  amount  of 
information  for  the  interval  of  ability  of  our  interest,  those 
of  Subtests  1  and  3  decline  au  the  level  of  ability  diverts  from 
this  area  in  either  the  negative  or  positive  direction,  w;:th  the  degree 
of  reduction  substantially  higher  for  Subtest  3.  It  is  recalled 


...  -■  .-x.  ■*•*-■■*.***** 
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FIGURE  2-1 


,1/2 


Square  Root  of  the  Test  Information  Function,  [1(9)1 
(Solid  Curve)  of  Subtest  J  and  the  Polynomial  of  Degree  7 
(Dotted  Curve) ,  Which  Was  Fitted  by  the  Method  of  Moments 
with  [-4.0,  4.0)  as  the  Interval  of  9  ,  Together  with 
the  Horizontal  Line  (■  4.65)  Which  Indicates  the  Square 
Root  of  the  Test  Information  Function  of  the  Original  Old 
Test.  Square  Root  of  the  Test  Information  of  Subtest  1 
Is  Also  Drawn  by  a  Dashed  Curve. 
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(Samejima,  RR-80-4)  that  none  of  the  five  hundred  maximum  likelihood 
estimates  of  ability  0  ,  which  were  obtained  upon  Subtest  1  for 
the  five  hundred  hypothetical  examinees  by  the  Monte  Carlo  method, 
assumes  negative  or  positive  infinity.  This  is  due  to  the  fact 
that  at  both  0  *  -2.5  and  0  *  2.5  ,  which  are  the  two  endpoints 
of  the  interval  for  the  uniform  distribution,  the  square  root  of 
the  test  information  function  of  Subtest  1  is  almost  as  high  as 
4.00  .  In  contrast  to  this,  the  square  root  of  the  test  information 
function  of  Subtest  3  at  0  -  -2.5  is  as  low  as  2.20  ,  and  the  one 
at  0  ■  2.5  is  as  low  as  2.45  .  For  this  reason,  it  is  more  likely 
that,  upon  Subtest  3,  examinees  whose  ability  levels  are  close  to 
the  lower  endpoiiit  of  the  interval,  (-2.5,  2.5)  ,  obtain  V-min, 
or  the  response  pattern  which  consists  of  n  zeros,  and  those 
whose  ability  levels  are  close  to  the  higher  endpoint  of  the  interval 
get  V-max  ,  or  the  response  pattern  which  has  the  n  highest  item 


scores,  m  (g*l,2, . . . ,n)  .  In  practice,  we  observe  fourteen 
8 

out  of  the  five  hundred  examinees  whose  response  patterns  are 


V-min  ,  and  twelve  whose  response  patterns  are  V-max  .  Table 


2-2  presents  the  identification  number  and  the  ability  level  of 


each  of  these  twenty-six  hypothetical  examinees.  As  we  can  see  in 
this  table,  all  hypothetical  examinees,  except  for  one,  who  obtained 
negative  infinity  for  their  maximum  likelihood  estimates  of  ability, 

A 

0y  »  are  located  lower  than  -2.000  in  their  ability  levels,  and 
also  those  who  obtained  positive  infinity  as  their  maximum  likelihood 
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TABLE  2-2 


Identification  Number  and  Ability  Level  of  Each  of 
the  Fourteen  Hypothetical  Examinees  Who  Obtained 
V-min,  and  of  the  Twelve  Who  Obtained  V-raax  . 


ID 

e 

ID 

6 

i 

-2.475 

491 

2.025 

101 

-2.475 

193 

2.125 

201 

-2.475 

493 

2.125 

401 

-2.475 

294 

2.175 

2 

-2.425 

296 

2.275 

102 

-2.425 

397 

2.325 

202 

-2.425 

98 

2.375 

302 

-2.425 

198 

2.375 

303 

-2.375 

199 

2.425 

4 

-2.325 

299 

2.425 

108 

-2.125 

499 

2.425 

109 

-2.075 

300 

2.475 

210 

-2.025 

118 

-1.625 

rn.Lj,-  -'"-"--a 


estimates  have  ability  levels  higher  than  2.000  .  The  only  exception 
in  the  former  group  of  examinees  is  the  hypothetical  examinee  No.  118, 
whose  ability  level  is  -1.625  ,  i.e.,  substantially  higher  than 
-2.000  ,  and  yet  whose  response  pattern  is  V-min  .  Eight  out  of 
the  fourteen  examinees  of  the  former  group  have  either  -2.425  or 
-2.475  for  their  maximum  likelihood  estimates,  and  seven  out  of 
twelve  of  the  latter  group  are  located  at  8  -  2.375  or  higher 
ability  levels. 

It  has  been  found  out  (Samejima  and  Livingston,  RR-79-2)  that 
the  method  of  moments  for  fitting  a  polynomial  of  a  specified  degree 
to  any  given  function  provides  us  with  one  which  is  also  the  least 
squares  solution  in  approximating  the  function  by  a  polynomial  of  the 
same  degree.  The  coefficients  of  such  a  polynomial  of  the  given 
degree,  m  ,  are  determined  solely  by  the  first  (m+1)  moments,  i.e., 
the  0-th  through  m-th  moments,  about  the  midpoint  of  the  selected 
interval  of  the  independent  variable  for  which  the  moments  were 
computed,  and  the  width  of  that  interval  itself.  It  has  also  been 
observed  that  the  goodness  of  fit  of  the  polynomial  to  the  given 
function  depends,  heavily,  upon  the  selected  interval,  as  well  as 
the  degree  of  the  polynomial,  m  . 

The  interval  of  0  chosen  for  approximating  the  square  root 

of  the  test  information  function  of  Subtest  3  is  (-4.0,  4.0)  ,  and 

the  degree  of  the  polynomial  is  seven.  Table  2-3  presents  the 

7  k 

coefficients  of  the  resultant  polynomial  of  degree  7,  or  £  6  , 
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table  2-3 

Coefficient*  of  the  Polynoaial  of  Degree  7 
Obtained  by  th*  Method  of  Monent*  Uaing 
the  Interval  of  6  ,  (-4.0,  4.0)  ,  to 

Approxiaate  th*  Square  Root  of  the  Teat 
Intonation  Function  of  Subteat  3. 


k 

“k 

0 

0. 46408884 D+01 

1 

0.607896S9D-01 

2 

-0.41482735D+00 

3 

0.1468403*0-01 

4 

0.S1686862D-02 

5 

-0.369C3316D-02 

■Klip 

0.21313602D-03 

1 

0.15726020D-03 

TABLE  2-4 

Coefficient*  of  th*  Polynoaial  of  Degree  8  to 
Trantfora  6  to  x  for  Subteat  3. 


k 

* 

0 

O.OOOOOOOOIHOO 

1 

0.13259652D+01 

2 

0.86842420D-02 

3 

-0.39306409D-01 

4 

0.10489276D-02 

S 

0.29536370D-03 

6 

-0. 17372918D-03 

7 

0.86989735D-05 

8 

0.36164139D-05 
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and  the  polynomial  Itself  is  drawn  by  a  dotted  curve  in  Figure  2-1. 

Ve  can  see  that  our  choice  of  the  degree  of  the  polynomial  and  that 
of  the  interval  of  6  have  resulted  in  an  extremely  good  approximation 
to  the  square  root  of  the  test  information  function  of  Subtest  3. 

It  has  been  shown  (Samejima,  RR-80-2)  that,  for  any  given 
test,  the  transformation  of  latent  trait  6  to  another  latent  trait, 

t  ,  which  provides  us  with  a  constant  test  information  ^unction, 

2 

I*(t)  ”  C  ,  for  the  interval  of  t  of  our  interest,  can  be  obtained 
from  the  polynomial  approximating  tne  square  root  of  the  test 
information  function  of  the  test.  Thus  we  can  write 


(2.12) 

m+1 

t  i  E  a*  9k  , 

lr  9 

k-0 

where 

d 

for 

(2.13) 

a£  i 

k  v-  (Ck)_1  a  1 

for 

2 

where  d  is  an  arbitrarily  set  constant  and  C  is  the  desired 

constant  amount  of  test  information  of  the  given  test  for  the 

transformed  latent  trait,  t  .  For  our  purpose,  we  have  used  d  -  0 

and  C  ■  3.5  for  Subtest  3.  Table  2-4  presents  the  coefficients  of 

the  resultant  polynomial  of  degree  8  for  transforming  ability  6  to 

T  ,  which  makes  the  square  root  of  the  test  information  function, 

1/2 

,  of  Subtest  3  approximately  equal  to  3.5  ,  for  the 
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Interval  of  x  ,  (-3.16466,  3.27619)  .  Figure  2-2  presents  the 

true  values  of  the  square  root  of  the  test  information  function  of 
Subtest  3  by  a  dotted  curve,  which  is  obtained  by 

(2.14)  (I*(x)]1/2  -  [i(e)]1/2  ~- 

1/2  m  k  -1 

-  [1(e) ]i/z  c  (  z  o.  e  j 

k-0 

together  with  the  horizontal  line  indicating  C  -  3.5  .  We  can  see 
in  this  figure  that  the  approximation  is  extremely  good,  as  is 
expected  from  Figure  2-1  . 

It  has  been  observed  (Samejima,  RR-79-3)  that,  using  equivalent, 
binary  items  following  the  Constant  Information  Model  (Samejima, 
RR-79-1) ,  the  speed  of  convergence  of  the  conditional  distribution 
of  the  maximum  likelihood  estimate,  given  ability,  to  the  normality 
is  not  constant,  but  is  substantially  different  depending  upon  the 
fixed  ability  level,  even  if  the  amount  of  test  inlormation  is 
constant  across  the  ability  levels.  We  should  expect,  therefore, 
that,  in  the  present  situation,  the  goodness  of  fit  of  the  normality, 
with  t  and  C  *  (-  0.285714)  as  the  two  parameters,  to  the 
conditional  distribution  of  the  maximum  likelihood  estimate,  given 
the  transformed  ability  t  ,  also  depends  upon  the  fixed  value  of 
t  .  This  fact  is  confirmed  from  the  fact  that,  outside  of  the 
interval  of  t  ,  (-2.30473,  2.38816)  ,  which  corresponds  to  the 
interval  of  0  ,  (-2.0,  2.0)  ,  we  have  observed  thirteen  hypothetical 


-40  -3.0  -2.0  -1.0  0.0  1.0  2.0  3.0  4.0 

LATENT  TRAIT  T 


FIGURE  2-2 

Square  Root  of  Test  Information  Function  of  Subtest  3 
Resultant  from  the  Polynomial  Transformation  of  6  to 
t  (Dotted  Line),  and  the  Target  Constant  Amount  of 
3.5  (Solid  Line). 
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examinees  whose  response  patterns  are  V-min  ,  or  the  set  of  n 

zeros,  and  twelve  examinees  who  have  V-max,  or  the  set  of  n  o  's  , 

8 

for  their  response  patterns.  Obviously,  the  convergence  to  the 
normality  based  upon  Subtest  3  is  slow  for  {.hese  deviated  ability 
levels.  For  this  reason,  it  is  necessary  that  we  use  some  other 

A 

astimv.te  of  t  than  the  maximum  likelihood  estimate  tv  for  each 
of  the  two  extreme  response  patterns,  V-min  and  V-max  ,  so  that 
the  resultant  conditional  distribution  of  the  estimate,  given  x  , 
be  approximately  normal  with  x  and  C  ^  as  the  two  parameters. 

One  solution  for  this  problem  is  to  use  the  second  modified  maximum 
likelihood  estimate,  x*  ,  which  was  introduced  in  a  previous  study 
(Samejlma,  RR-81-1) .  This  estimate  is  defined  by 


(2.15) 


x$  < 


‘V-min 


TV-i 


max 


for  V  -  V-min 


for  V  ■  V-max 


otherwise  , 


with  x*  .  and  x*  having  such  mathematical  forms  as 

V-min  V-max 


(2.16) 


TV-min  ’  [2(Tc  +  ^  \  ~  ^  "lv1  NbV-min 

V/ V-max 


T*  -  [i(i  +  x  )  N  —  E  X  N  )  N  -1 
V-max  2  1  c  H  y4y^in  V  HV  HV-max 


V»*V-max 
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where  x  and  x  are  the  lower  and  upper  endpoints  of  the  interval 

of  x  for  which  Subtest  3  is  considered  to  be  effective,  t  is  the 

c 

critical  value  of  x  below  which  the  operating  characteristic, 

P*_  x(x)  ,  of  t  e  response  pattern  V-max  assumes  negligibly 

small  values  and  above  which  so  does  the  operating  character istic, 

P*_min(T)  ,  of  the  response  pattern  V-min  ,  and  are  the 

sample  sizes  of  the  lower  and  the  upper  ability  groups  which  were 

separated  by  the  critical  value,  tc  ,  respectively,  and  NLV  and 

are  the  numbers  of  examinees  who  belong  to  the  lower  ability 

group  and  have  obtained  a  specific  response  pattern  V  ,  and  who 

belong  to  the  higher  ability  group  and  have  obtained  V  ,  respectively. 

This  modified  maximum  1 ikelihood  estimate  is  the  sample  statistic 

version  of  the  first  modified  maximum  likelihood  estimate,  x*  , 

(Samejima,  RR-80-3,  RR-81-1)  ,  and  is  useful  when  the  number  of  all 

possible  response  patterns  of  a  given  test  is  too  large  for  the 

computation  o.  x*  .  An  important  characteristic  of  the  modified 

maximum  likelihood  estimate,  t*  ,  and  that  of  x^  ,  is  that,  with 

a  suitable  choice  of  the  interval,  (x,x)  ,  the  estimate  is, 

approximately,  conditionally  unbiased,  as  asymptotically  is  the  case 

with  the  maximum  likelihood  estimate.  In  order  to  obtain  x£  . 

V-min 

«*w 

and  Ty.max  *  which  are  defined  by  (2.16),  we  must  prepare  a  large 
size  sample  from  the  uniform  distribution  of  x  for  hhe  interval, 

(x,x)  ,  and  then  produce,  by  the  Monte  Carlo  method,  a  response 
pattern  for  each  hypothetical  examinee  upon  the  test  in  question. 
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With  a  suitable  selection  of  the  interval.  (x,x)  ,  we  may  be 
successful  in  obtaining  ^y_njn  and  *v-max  whic^  approximate  the 
conditional  distribution  of  x*  ,  given  t  ,  to  the  normality  with 
t  ind  C  ^  as  the  tv*o  parameters. 

Using  the  modified  maximum  likelihood  estimate,  T*  ,  instead 
of  the  maximum  likelihood  estxamte,  x^  ,  we  can  proceed  to  the 
eatimation  of  the  operating  characteristics  of  the  discrete  item 
responses  of  a  new  test  item,  using  such  approaches  as  Histogram 
Ratio  Approach,  Curve  Fitting  Approach,  Conditional  P.D.F.  Approach, 
which  Includes  Simple  Sum  Procedure,  Weighted  Sum  Procedure  and 
Proportioned  Sum  Procedure,  and  Bivariate  P.D.F.  Approach,  each  of 
which  is  combined  with  Two-Parameter  Beta  Method,  Pearson  System 
Method  or  Normal  Approach  Method,  and  so  forth.  The  outlines  of 
these  procedures  are  described  in  a  previous  study  (Samejlma,  RR-80-2). 
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III  Selection  of  the  Interval,  (x,x)  ,  and  the  Critical  Value  t 
in  Obtaining  x*  c 

We  can  write  for  the  conditional  expectation  and  variance  of 

the  tiodified  '"iximum  likelihood  esti\na:e,  x*  ,  given  ,  . 

(3.1)  E(t*|t)  -  E  x*  P*(t) 
and 

(3.2)  Var.(T*|x)  -  E  [x*  -  E(t*{t) ]2  p*(x)  , 

where  P*(x)  is  the  operating  characteristic  of  the  response  pattern 
V  defined  with  respect  to  the  transformed  latent  trait  t  ,  and 
satisfies 

(3.3)  P*(t)  -  Py[0(T)]  . 

It  is  noted  from  (3.1)  and  (3.3)  that,  as  t  becomes  less,  the 

A  A 

conditional  expectation  of  x*  tends  to  x*  .  From  this  fact 

V  V-min 

and  (3.2),  it  is  further  noted  that  the  conditional  distribution  of 

*  A  A 

t*  ,  given  x  ,  approaches  a  one-point  distribution  at  x*  -  x*  , 
as  x  becomes  less.  Following  a  similar  logic,  we  note  that  the 
conditional  distribution  of  x*  ,  given  x  ,  approaches  a  one-point 
distribution  at  x^  ■  x*  max  as  t  grows  larger.  This  fact  implies 
that,  if,  for  the  response  pattern  V-min  ,  we  use  some  substitute 
estimate  which  is  higher  than  the  lowest  finite  value  of  the  maximum 
likehood  estimate  with  respect  to  a  given  test,  or  if,  for  the  response 
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pattern  V-max  ,  * r  use  some  substitute  which  is  lower  than  the  highest 
finite  value  of  the  maximum  likelihood  estimate,  the  regression  of 
the  estimate  on  t  cannot  be  a  strictly  .‘ncreaaing  function  of  i  . 

We  may  conclude,  therefore,  that  such  a  substitute  estimate  is  not 
desirable,  unless  there  is  a  good  reason  for  choosing  one. 

We  can  easily  see  that,  in  such  models  as  the  normal  ogive 
model  and  the  logistic  model,  etc.,  the  lowest  finite  value  of  the 
maximum  likelihood  estimate  belongs  to  one  of  the  n  response  patterns 
of  the  type,  (0,0, . . , ,1, . . . ,0)  ,  and  the  highest  finite  value  belongs  to 
one  of  the  n  response  patterns  of  the  type,  (rn^rn^, . . . ,m^-l, . . . .m^)  . 
Table  3-1  presents,  for  Subtest  3,  the  fifteen  response  patterns  of  the 
former  type,  and  the  two  maximum  likelihood  estimates,  6V  and  xv  , 
the  latter  of  which  was  obtained  by  (2.12)  with  the  substitution  of 
6y  for  6  ,  for  each  of  the  fifteen  response  patterns.  From  this 
table,  we  can  see  that  the  lowest  finite  maximum  likelihood  estimate. 


ty  ,  is  -2.6518  ,  and  the  highest  finite  maximum  likelihood  estimate 

is  2.7683  .  We  can  conclude,  therefore,  that  it  is  desirable  to 

choose  an  interval,  (t,t)  .  vhich  provides  us  with  t*  .  and 

—  v-Din 

r*  ,  the  former  of  which  is  less  than  -2.6518  and  the  latter 
v— max 

of  which  is  greater  than  2.7683  . 

There  is  another,  somewhat  opposing  factor  that  we  must  take 
into  consideration,  however.  Although  we  may  like  to  conclude  that  a 
given  test  is  effective  for  a  wide  range  of  ability,  for  the  present 
purpose  of  using  Subtest  3  as  the  Old  Test  for  estimating  the  operating 
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characteristics  of  the  discrete  item  responses  of  unknown  test  items, 

the  approximate  conditional  unbiasedness  of  the  estimate  is  not 

sufficient.  What  we  need,  in  addition,  is  the  approximate  normality 

of  the  conditional  distribution  of  the  estimate,  given  ability,  with 

C  ^  as  the  secono  parameter.  Considering  the  fact  that  the 

conditional  variance  of  x*  ,  given  x  ,  tends  to  zero  as  x  becomes 

less,  and  also  as  x  grows  greater,  the  choice  of  too  wide  an  interval 

must  be  avoided,  even  if  the  approximate  unbiasedness  of  the 

conditional  distribution  of  x*  still  holds  for  that  interval. 

Figure  3-1  presents  the  two  operating  characteristics, 

P*  .  (x)  and  P*  (x)  ,  by  solid  and  dotted  curves,  respectively, 
v  “tain  V“Oflx 

As  we  can  see  in  this  figure,  outside  of  the  interval  of  x  , 

(-3.0,  3.0)  ,  either  one  of  these  two  operating  characteristics 
becomes  greater  than  0.8  ,  the  fact  which  indicates  how  s  'eedy  the 
convergence  of  the  conditional  distribution  of  x*  ,  given  x  ,  to 
each  one-point  distribution  is.  From  this  figure,  we  must  say  that, 
even  outside  of  a  smaller  interval,  (-2.8,  2.8)  ,  either  one  of  the 
two  conditional  probabilities  for  the  response  patterns,  V-min  and 
V-max  ,  is  too  large. 

We  have  observed  in  a  previous  study  (Samejima,  RR-81-1)  eight 

different  cases  of  the  set  of  the  estimates,  x*  .  and  x*  , 

V-min  V-max 

upon  Subtest  3,  which  were  obtained  by  using  eight  different 
intervals  for  (x,x)  .  The  critical  value,  xc  ,  which  we  used  in 
obtaining  these  estimates,  is  -0.5455  ,  and  the  values  of 


^Tl’il  -aiA. 


-24- 


III-4 


above  this  point  of  t  are  less  than  0.00000001,  and  those  of 

P*  (t)  below  it  less  than  the  same  value,  which  satisfy 
V-max 

the  requirement  (Samejima,  RR-80-3)  that  these  values  be  negligibly 

small.  This  value  of  x^  is  more  or  less  arbitrary,  i.e.,  only 

one  of  the  infinitely  many  values  of  t  which  satisfy  the  above 

requirement,  however.  As  another,  probably  more  meaningful,  value 

of  ,  here  we  take  the  value  of  Tc  at  which  the  product  of  the 

two  operating  characteristic,  P*  .  (t)  and  P*  (t)  ,  becomes 

maximal.  This  value  of  x^  is  also  the  polynomial  function  of 

8c  ,  whose  coefficients  are  given  by  Table  2-4,  with  9  -  ©c 

being  the  value  of  the  original  ability  9  at  which  the  product 

of  the  two  operating  characteristics,  Ptt  .  (0)  and  P„  (0)  , 
r  V-min  V-max 

assumes  the  maximal  value.  It  turned  out  that  0  *  0.0907  and 

c 

T  «  0.1203  .  The  positions  of  these  two  values  of  x  are 
c  c 

indicated  by  two  arrows  in  Figure  3-1.  The  values  of  P*  .  (t) 

V  "“in  in 

for  all  points  of  x  above  the  critical  value,  0.1203  ,  are, 

again,  less  than  0.00000001  ,  and  so  are  those  of  P*  (x)  for 
°  V-max 

x  <  0.1203  .  In  fact,  this  is  true  with  any  value  of  x  in  the 
interval,  (-0,91,  1.05)  ,  in  which  both  -0.5455  and  0.1203  are 
included . 

A 

Table  3-2  presents  the  resultant  estimates,  x*  ,  and 

V-min 

X*  max  ,  obtained  by  using  each  of  the  eight  intervals,  together 
with  the  sample  sizes,  N  ,  N  and  N  (*  N  +  N  )  ,  and  the 
two  frequencies,  Nv_min  and  Nv-raax  ’  7or  colnParison»  Table  3-3 
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presents  the  corresponding  results  (Samejima,  RR-81-1)  obtained  by 

using  x^  ■  -0.5455  and  each  of  the  same  eight  intervals  of  t  . 

As  we  can  see  in  these  two  tables,  the  two  frequencies, 

and  N,.  ,  are  too  small  in  the  first  three  cases  and  the  results 

V-max 

should  not  be  taken  seriously. 

Comparison  of  the  two  sets  of  results  for  each  of  the 

remaining  five  cases,  which  are  shown  in  Tables  3-2  and  3-3,  indicates 

that,  for  each  interval  of  t  ,  the  resultant  set  of  x*  .  and 

V-min 

a 

TV-oax  are  very  cl°se  to  each  other.  There  is  a  slight  tendency 

that  these  values,  which  were  obtained  by  using  t  ■  0.1203  ,  are 

c 

greater  in  absolute  values  than  those  obtained  by  using  x^  *  -0.5455  , 
but  the  differences  are  not  so  great,  i.e.,  approximately  between 
0.022  and  0.210  .  There  is  a  tendency  that  these  discrepancies 
become  less  as  the  interval,  (x,x)  ,  becomes  larger,  or  the 
frequencies,  and  become  greater.  In  fact,  for 

the  interval,  (-3.0,  3.0)  ,  the  discrepancy  between  the  two 

a 

TV-min  '8  *s  as  8ma11  as  -0.0218  ,  and  the  one  for  the  two 

A  A 

x*  s  is  0.0259  .  The  sample  mean  and  variance  of  t*  for 

V-max  r  V 

each  of  the  five  cases  and  the  sample  correlation  coefficient  of 

A 

x*  and  x  are  given  in  Table  3-4,  for  the  two  situations  in 

which  t  *  0.1203  and  x  ■  -0.5455  ,  respectively.  In  the  same 
c  c 

table,  also  presented  in  brackets  are  the  theoretical  mean  and 
variance  of  an  estimator  which  is  conditionally  unbiased,  given 
x  ,  and  whose  conditional  distribution  is  N(x,C~*)  ,  where 


TABLE  3-4 


r 
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C  •  3.5  .  Lee  X  denote  such  an  estimator.  We  can  write 

(3.4)  E(X)  -  E(t)  , 
and 

(3.5)  Var.(X)  -  Var.(x)  +  (f2  . 

The  correlation  coefficient  between  X  and  x  is  given  by 

(3.6)  Corr . (x , X)  -  [1-C-2 • (Var. (X) }_1 ]1/2  . 

This  value  is  also  presented  in  brackets  in  Table  3-4,  for  each  of 
the  five  intervals  of  x  . 

We  can  see  in  this  table  that  the  results  obtained  by  using 

x  ■  0.1203  are  very  close  to  those  obtained  by  using  x  *  -0.5455 
c  c 

We  notice,  however,  that  all  these  values  in  the  former  situation 
are  closer  to  the  expected  population  parameters  obtained  with  X  , 
although  the  differences  are  small. 

Table  3-5  presents  the  sample  linear  regression  coefficients 
of  {•*  on  x  ,  which  is  given  by  ax  +  f}  ,  for  each  of  the  five 
cases  and  in  each  of  the  two  situations.  As  is  expected,  the  two 
sets  of  results  are  very  similar.  There  is  a  slight  tendency, 
however,  that  the  values  of  a  are  closer  to  unity,  and  those  of 

6  are  closer  to  tero,  in  the  former  situation  where  x  ■  0.1203  . 

c 

When  we  take  all  the  observations  we  made  in  the  preceding 
paragraphs,  perhaps  the  best  choice  of  the  interval,  (x,x)  ,  and 


m  '  liHli  mm mm 


... 
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TABLE  3-5 

Two  Coefficients  of  the  Sample  Linear  Regression  of 
x*  ,  Which  Was  Obtained  upon  Subtest  3,  on  t  ,  for 

Each  of  the  Five  Intervals  of  t  in  Each  of  the  Two 

Situations,  Where  T  -  0.1203  and  t  -  -0.5455  , 

c  c 

Respectively. 


Case 

T  “  ( 
C 

).1203 

T  »  - 
C 

0.5455 

a 

8 

a 

8 

4 

0.99849 

0.00096 

0.99588 

0.00111 

5 

0.99868 

0.00038 

0.99605 

0.00057 

6 

0.99797 

-0.00026 

0.99542 

-0.00022 

7 

0.99892 

0.00032 

0.99673 

0.00053 

8 

0.99795 

_ 

0.00016 

0.99599 

0.00027 

111-7 


the  critical  value,  xfi  ,  from  our  available  data  will  be 

(-3.0,  3.0)  and  0.1203  .  This  is  the  only  interval  which 

provides  us  with  which  is  less  than  the  least  finite 

maximum  likelihood  estimate,  -2.6518  ,  of  Subtest  3,  and  with 

max  which  is  greater  than  the  greatest  finite  maximum 

likelihood  estimate,  2.7683  ,  in  each  of  the  two  situations  where 

x  «*  0.1203  and  t  =  -0.5455  ,  respectively.  For  the  purpose  of 
c  c  ' 

illustration,  the  sample  regression  of  on  t  ,  which  is  based 

upon  the  interval,  (-2.430,  2,586)  ,  and  x£  *  -0.5455  ,  is  shown 
for  tue  interval  of  x  ,  (-3.0,  3.0)  ,  in  Appendix  as  Figure  A-l. 
Although  this  is  a  sample  regression  based  upon  one  thousand 
equally  spaced  points  of  x  with  five  observations  at  each  point 
(Samejima,  RF-81-1) ,  a  similar  S-shape  is  also  expected  in  the 
population  regression.  Although  this  example  is  a  little  extreme, 
a  similar  tendency  will  be  seen  if  we  use  one  of  the  results  which 
are  based  upon  the  four  intervals  other  than  (-3.0,  3.0). 


(3.7) 


The  error  score,  eg  ,  which  is  defined  by 


es  =  [TV  -  Ts]  ri*(Ts)} 
s 


where  s  denotes  an  individual  hypothetical  examinee  and  Vg  and 

x  are  his  response  pattern  and  ability  level,  respectively,  was 
s 

computed  for  each  of  the  5,000  hypothetical  examinees  using 


T* 

V-min 


-2,8432  and  x*  *  2.8855  ,  which  were  obtained  by 
V-max 


using  x£  -  0.1203  .  Since  [I*(t)}  ' "  »  3.5  for  Subtest  3,  this 


constant  value  was  used  In  (3,7)  for  the  above  computation.  For 

comparison,  the  error  score  Is  also  computed  for  the  A, 180 

hypothetical  examinees,  using  x*  ■  -1.8162  and  ■  2.2439 

which  were  obtained  by  using  the  same  value  of  . 

Figures  3-2  and  3-3  present  the  frequency  distributions  of 

these  two  sets  of  error  scores,  eg  ,  respectively,  which  were 

constructed  with  the  category  width  of  0.2  ,  together  with  the 

standard  normal  density  function.  The  chi-square  test  for  the 

goodness  of  fit  of  each  of  these  two  frequency  distributions  against 

the  standard  normal  distribution  was  performed  by  categorizing  all 

the  subintervals  below  e  ■  -2.8  into  one  class  and  all  above 

2 

e  -  2.8  into  another.  As  the  results,  we  obtained  Xq  *  44.281 
2 

and  Xq  -  25.573  with  29  degrees  of  freedom  each,  which  provide 
us  with  0.025  <  p  <  0.050  and  0.50  <  p  K  0.70  ,  respectively. 

From  all  aspects,  it  may  be  feasible  to  adopt  -2.843  and 
2.885  as  Ty_min  and  Ty-max  *  resPective^y •  The  corresponding 
values  of  6  to  these  two  values  of  t  are  -2.F08  and  2.641  . 
Note,  however,  that  these  two  values  of  6  are  not  the  same  as 
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IV  Estimation  of  the  Item  Characteristic  Functions  of  Ten  Binary 
Test  Items  Using  Subtest  3.  As  the  Old  Test 

We  shall  proceed  to  use  Subtest  3  as  the  Old  Tmt  in  the 
process  of  estimating  the  operating  characteristics  of  the  discrete 
responses  of  unknown  test  items.  Our  simulated  data  are  based  upon 
five  hundred  hypothetical  examinees  whose  ability  levels  on  the 
original  latent  trait  9  are  distributed  over  one  hundred  equally 
spaced  positions  in  the  interval  of  6  ,  (-2.5,  2.5)  ,  with  five 
examinees  placed  at  each  position,  as  we  have  used  them  repeatedly 
in  our  previous  studies  (Samejima,  1977,  RR-77-1,  RR-78-1 ,  RR-78-2, 


RR-78-3,  RR-78-4,  RR-78-5,  RR-78-6,  RR-80-2,  RR-80-4).  They  are 
considered  as  a  sample  representing  the  uniform  distribution  of  6 
for  the  Interval,  (-2.5,  2.5)  .  This  uniform  density  function  is 
drawn  by  a  dotted  line  in  Figure  4-1  When  6  is  transformed  to 
t  by  (2.13)  with  the  coefficients  shown  in  Table  2-4,  the  ability 
distribution  is  no  longer  uniform,  but  its  density  function  is  of 
a  U-shape,  which  is  drawn  by  a  solid  line  in  Figure  4-1. 

The  difference  of  the  present  procedure  of  using  Subtest  3 

from  the  one  in  which  we  used  either  Subtest  1  or  Subtest  2  (Samejima, 

RR-80-4)  is  that  the  modified  maximum  likelihood  estimate,  f*  , 

is  used  in  place  of  the  maximum  likelihood  estimate,  .  In  so 

doing,  we  define  t*  .  and  t*  such  that 

V-min  V-max 


(4.1) 


T*  .  »  -2.843 

V-min 


t*  -  2.885  , 

V-max 


following  the  result  obtained  by  using  the  interval  of  t 


<ia 


A11SN30 


Theoretical  Density  Function  of  the  Original  Latent  Trait  0  (Dotted  Line) 
and  That  of  the  Transformed  Latent  Trait  t  (Solid  Line) . 
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(-3.0,  3.0)  ,  which  we  observed  in  the  preceding  chapter.  The 
resultant  x*  '  s  for  the  five  hundred  hypothetical  examinees 
are  plotted  against  x  for  the  five  hundred  hypothetical  examinees 
in  Figure  4-2.  The  sample  linear  regression  of  x*  on  x  turned 
out  to  be  1.01213x  -  0.00439  ,  which  is  close  to  the  straight 
line  with  forty-five  degrees  from  the  abscissa  passing  the  origin, 
(0,0)  ,  and  is  shown  in  the  same  figure.  The  sample  mean  and 
the  standard  deviation  of  the  five  hundred  x*  's  are  0.01698 
and  1.75384  ,  respectively,  and  the  sample  product-moment 
correlation  coefficient  between  x  and  x*  is  0.987  . 

The  bivariate  density  function,  £*(x*,x)  »  of  T  and 
x*  is  given  by 

(4.2)  S*(x*,x)  -  i|>*(x^|x)  f*(x)  , 

where  i|»*(x*|x)  is  the  conditional  density  function  of  x*  ,  given 
x  ,  and  f*(x)  is  the  marginal  density  function  of  x  .  We  can 
write  for  the  marginal  density  function,  g*(x^)  ,  of  , 

(4.3)  g*(x*)  *  ^  5*(x*,x)  dx 

Figure  4-3  presents  this  theoretical  density  function  by  a 
thick,  solid  line,  which  was  obtained  by  assuming  that  x*  is 
unbiased,  and  its  conditional  distribution,  given  x  ,  is  normal 
with  C  *  as  the  second  parameter.  Note,  however,  that,  in  reality, 
this  assumption  is  only  approximately  satisfied.  In  the  same  figure, 
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also  presented  is  a  histogram  which  represents  the  relative  frequency 
distribution  of  the  five  hundred  t*  's  ,  using  the  interval  width 
of  0.25  . 

It  is  noted  in  this  figure  that  both  the  lower  and  upper  ends 
of  the  histogram  are  rather  abrupt,  with  no  tails.  For  comparison, 
the  corresponding  histogram  and  marginal  density  function,  which 
are  based  upon  Subtest  1,  of  which  Subtest  3  is  a  subset,  is  shown 
as  Figure  4-4.  We  can  see  that,  for  Subtest  1,  the  histogram  has 
tails  in  both  the  negative  and  positive  directions.  The  reason 
for  this  difference  is  that,  for  Subtest  3,  there  are  certain 
numbers  of  examinees  whose  maximum  likelihood  estimates  are  negative 
and  positive  infinities,  respectively,  and  they  were  uniformly 
replaced  by  two  finite  numbers.  The  error  score,  eg  ,  which  Is  defined 
by  (3.7),  was  computed  for  each  of  the  five  hundred  hypothetical 
examinees,  and  is  presented  in  Figure  4-5  in  the  form  of  a  histogram 
with  0.20  as  the  category  width,  together  with  the  standard  normal 
density  function,  which  is  drawn  by  a  dotted  line.  The  chi-square 
test  for  the  goodness  of  fit  was  performed,  and  we  obtained 
Xq  *  28.68328  ,  with  29  deg.tees  of  freedom,  which  provides  us 
with,  approximately,  p  *  0.50  . 

The  set  of  unknown  test  items  consists  of  ten  binary  items, 

each  of  which  follows  the  normal  ogive  model,  whose  item 

characteristic  function  is  given  by  (2.2)  with  m  ■  1  and  for 

6 

x  ■  1  .  Table  4-1  presents  the  item  discrimination  parameter, 
a^  ,  and  the  item  difficulty  parameter,  b^  ,  of  each  of  the  ten 
new  binary  items,  h  (**1,2 . 10)  . 
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We  can  write  for  the  conditional  density  function  of  t  , 
given  T*  ,  which  is  denoted  by  d>*(  t  j  f*)  ,  such  that 

(4. A)  ♦*<t|t*)  -  £*(t'*,t)  tg*(f*)]_1  . 

In  the  Simple  Sum  Procedure  of  the  Conditional  P.D.F.  Approach, 

this  conditional  density  takes  an  essential  role  in  estimating  the 

operating  characteristics  of  the  discrete  item  responses  of  unknown 

test  items.  Let  f*  be  a  simplified  version  of  ,  i.e.,  the 

s 

modified  maximum  likelihood  estimate  of  the  ability  t  of  the  examinee 

s  («1,...,N).  We  can  write  for  the  criterion  operating  characteristic, 

R  (6)  ,  of  the  discrete  item  response  x.  of  the  unknown  item  h 
xh  h 

N  , 

(4.5)  R  (8)  -  R*  [  T  (0)  ]  -  Z  d»*(T  K*)  [  l  1  , 

*h  h  sex.  s**l 

n 

where  sex.  indicates  an  examinee  s  whose  response  to  item  h  is 
n 

.  In  practice,  since  the  marginal  density  function  f*(0)  is 

not  observable,  R  (8)  is  not  observable,  either.  With  empirical 
Xh 

data,  we  need  to  estimate  the  conditional  density  function,  $*(1)7^)  , 
and  this  is  done  by  using  the  method  of  moments  (Elderton  and  Johnson, 
1969)  effectively.  With  our  simulated  data,  however,  (4.5)  can  be 
computed  directly,  and  used  as  a  criterion  for  evaluating  the  specific 
method  adopted  in  our  study.  The  name,  criterion  operating 
characteristic  came  from  this  fact,  and  its  availability  is  one  of 
the  reasons  why  the  Monte  Carlo  study  is  valuable. 

Figure  4-6  presents  the  criterion  operating  characteristic 
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for  the  correct  answer,  or  x.  *  1  ,  or  the  criterion  item 

n 

characteristic  function,  of  each  of  the  ten  unknown,  binary  test 
items,  by  a  dotted  line.  In  the  same  figure,  also  presented  are 
two  other  criterion  item  characteristic  functions,  which  were 
obtained  upon  the  original  Old  Test  and  Subtest  1,  by  longer  and 
shorter  dashed  lines,  respectively,  together  with  the  theoretical 
item  characteristic  function,  which  is  drawn  by  a  solid  line,  for 
each  binary  test  item.  As  we  have  observed  in  a  previous  study 
(Samejima,  RR-80-4) ,  for  each  item,  the  two  criterion  item 
characteristic  functions  based  upon  the  Old  Test  and  Subtest  1  are 
practically  the  same,  the  fact  that  we  can  confirm  in  Figure  4-6.  We 
notice,  further,  that  the  criterion  item  characteristic  function, 
which  is  based  upon  Sub test  3  ,  is  very  close  to  those  two  other 
criterion  item  characteristic  functions,  and,  more  importantly,  it 
is  close  to  the  theoretical  item  characteristic  function,  for  each 
and  every  binary  test  item.  Slight  discrepancies  are  observed, 
however,  for  farther  values  of  9  ,  i.e.,  discrepancies  for  items 
1,  3  and  4  for  the  range  of  0  less  than  -2.0  ,  and  those  for 
items  8,  9  and  10  for  the  range  of  6  greater  than  2.0  .  This 
is  anticipated  from  the  fact  that  the  amount  of  test  information  is 
substantially  less  for  Subtest  3  in  comparison  with  both  Subtest  1 
and  the  original  Old  Test,  for  these  ranges  of  0  ,  as  we  can  see  in 
Figure  2-1,  although  they  are  less  important  ranges  of  ability  for 
the  present  purpose . 

With  any  empirical  data,  we  must  use  an  estimate  of  the 
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conditional  density  function,  <J>*(t|t*)  .  To  obtain  the  estimate, 

the  conditional  moments  of  ?  ,  given  t*  ,  take  important  roles. 

8 

We  can  write  for  the  first  and  second  conditional  moments  of  t 

about  the  origin,  given  t*  ,  such  that 

s 

(4.6)  E(t  | f*)  *  t*  +  C'2  ^  log  g*(f*) 

s 

and 

(4.7)  E(x2 | f *)  «  t*  +  2t*  C  2  -jir  log  g*(x*) 

1  s  s  s  dt*  s 

s 

+  lo8  log  g*(T*)}2]  +  C 

s  Ts 


It  is  obvious  from  (4.6)  and  (4.7)  that  we  shall  be  able  to  obtain  the 
estimates  of  these  two  conditional  moments,  provided  that  we  can 

estimate  the  marginal  density  function,  g*(x*)  .  This  can  be  done 

8 

by  using  the  method  of  moments  for  fitting  a  polynomial,  which 
provides  us  with  the  least  squares  solution  (Samejima  and  Livingston, 
RR-79-2) . 

Table  4-2  presents  the  first  through  tenth  sample  moments 

of  t*  about  the  origin,  which  were  obtained  for  our  five  hundred 
s 

observations  of  t*  .  In  the  same  table,  also  presented  are  the 

corresponding  ten  sample  moments  about  the  midpoint  of  the  interval 

of  t*  ,  which  turned  out  to  be  0.021  .  This  second  set  of  moments 
s 

is  actually  used  for  obtaining  polynomials  following  the  method  of 
moments,  the  detailed  procedure  of  which  is  described  in  a  previous 
study  (Samejima,  RR-77-1) . 


TABLE  4-2 

First  Through  Tenth  Sample  Moments  of  t*  about  the 

Origin  Obtained  for  the  Five  Hundred  Observations, 
and  the  Corresponding  Sample  Moments  about  the 

Midpoint. 


Moments  About 
Origin 

Moments  About 
Midpoint 

1 

0.16976800D-01 

-0.40232000D-02 

2 

0 . 30762381D+01 

0.307596610+01 

3 

0. 32136059D+00 

0.12757078D+00 

4 

0.161321120+02 

0.16113257D+02 

5 

0.307934680+01 

0.13866074D+01 

6 

0. 99326410D+02 

0 . 99045076D+02 

7 

0.269329550+02 

0.12355264D+02 

8 

0.66322009D+03 

0.659920460+03 

9 

0. 22694745D+03 

0.101949240+03 

10 

0.465204590+04 

0.46175227D+04 

58- 


TABLE  4-3 


Coefficients, 


U) 


j 


of  the  Polynomials  of 


Degrees  3  Through  7  ,  Which  Approximate 
the  Density  Function,  g*(x*)  ,  and  Were 
Obtained  by  the  Method  of  Moments. 


j 

Coefficient  ai 

°  G 

1  R 

2  . 

3  3 

0.14752089D+00 

-0.10711228D-01 

0.98492052D-02 

0.20396181D-02 

0  D 

1  G 

2  R 

3 

4  4 

0.15724612D+00 
-0.10213053D-01 
-0 . 20091300D-02 
0.18978863D-02 
0.16872831D-02 

'  i 

3  . 

S  3 

0.15707407D+00 
-0.20242784D-02 
-0. 17154481D-02 
-0. 27622271D-02 

0. 16335685D-02 

0. 51156775D-03 

0 

1  D 

2  G 

3  R 

4 

5  6 

6 

0.13966552D+00 
-0. 38977430D-02 
0.42862067D-01 
-0.13915383D-02 
-0.14677720D-01 

0. 32771426D-03 

0. 1459154 7D-02 

0 

3  2 
l  ? 

1  3 

7 

0.13999730D+00 

-0.19681749D-01 

0.41769825D-01 

0, 15931960D-01 
-0.14189397D-01 
-0.43208192D-02 
0.14075468D-02 

0. 35107420D-03 

-59- 


IV- 7 


polynomials  of  degrees  3  through  7  ,  such  that 

(4.8)  g*(T*)  -  1  u>  ,  p  =  3, 4, 5, 6, 7  , 

S  j“0  ^  s 

which  were  obtained  by  using  up  to  the  p-th  sample  moments  about 
the  midpoint.  These  five  polynomials  are  shown  in  the  five  separate 
graphs  of  Figure  4-7,  together  with  the  frequency  distribution  of 
the  five  hundred  i*  's  .  These  estimated  density  functions,  , 

were  obtained  for  the  interval  of  t*  ,  [-2.843,  2.885]  .  When  we 

compare  these  five  curves  with  the  theoretical  density  function  of 
t*  ,  which  was  obtained  assuming  the  exact  unbiasedness  of  the 
estimate  and  the  perfect  normality  for  the  conditional  distribution 
of  the  estimate,  given  t  ,  with  C-1  (»  0.28571)  as  the  second 

parameter,  and  is  shown  in  Figure  4-3  ,  we  notice  the  similarity 
between  them.  We  also  note  a  difference,  however,  for  the  two 
extreme  ranges  of  t  ,  at  which  these  five  polynomials  go  up  instead 
of  coming  down,  as  we  can  see  in  the  theoretical  curve.  This 
tendency  becomes  more  conspicuous  as  the  degree  of  a  polynomial 
grows  larger. 

These  results  are  expected  from  the  fact  that  the  conditional 
distribution  of  the  modified  maximum  likelihood  estimate,  x*  , 
given  t  ,  is  truncated  for  the  values  of  t  in  these  two  extreme 
ranges,  as  is  indicated  in  Figure  4-2  ,  and  the  violation  from 
the  unbiased  normality  for  the  conditional  distribution  is 
substantial  for  these  ranges  of  x  .  How  these  discrepancies  of 
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FIGURE  4-7  (Continued) :  Polynomial  of  Degree 
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the  estimated  density  functions,  g*(?*)  ,  will  affect  the 
accuracy  of  estimation  of  the  item  characteristic  functions  of 
the  ten  unknown,  binary  test  items  is  yet  to  be  seen. 

As  we  have  done  in  a  previous  study  for  Sub tests  1  and  2 
(Samejima,  RR-80-4) ,  we  shall  use  the  polynomials  of  degrees  3  and 
4  ,  separately,  for  the  estimated  density  function,  g*(x*)  , 
and,  hereafter,  we  shall  call  the  former  case  as  Degree  3  Case, 
and  the  latter  Degree  4  Case.  The  estimate,  $*(t|t*)  ,  for  the 
conditional  density  function  of  t  ,  given  i*  ,  is  given  by 

(4.9)  ♦MtIt*)  -  [2ir]'1/2  exp[-(T-u)2/(202)]  , 


i 

1 

I 


where  y  is  the  estimate  of  the  first  conditional  moment,  E(t|t*)  , 

s 

and  o2  is  the  estimate  of  the  second  conditional  moment,  E(t2|t*)  , 

s 

subtracted  by  the  square  of  the  first  estimate,  which  were  obtained 

by  replacing  g*(x*)  in  (4.6)  and  (4.7)  by  g*(x*)  .  These  estimated 
s  s 

conditional  mean  and  variance  for  each  of  the  five  hundred  x*  's 

s 

are  presented  in  Appendix  as  Tables  A-l  and  A-2,  for  Degree  3  and 

4  Cases,  respectively.  From  the  estimated  conditional  density 

functions,  which  are  given  by  (4.9),  we  obtain  the  estimated 

operating  characteristic,  P  (0)  ,  of  the  discrete  item  response 

*h 

x,  of  an  unknown  test  item  h  by 
n 


(4.10)  f  (e)  =  f*  [  t( 0) )  =  Y.  $*(t|t*)  [  1  $*<t|t*)] 
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FIGURE  4-8  (Continued):  Degree  3  Case. 


FIGURE  4-8  (Continued):  Degree  3  Case. 


FIGURE  4-8  (Continued):  Degree  3  Case. 


FIGURE  4-8  (Continued):  Degree  3  Case. 


FIGURE  4-8  (Continued):  Degree  3  Case. 


FIGURE  4-9  (Continued):  Degree  4  Case. 


FIGURE  4-9  (Continued):  Degree  4  Case. 
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FIGURE  4-9  (Continued):  Degree  4  Case. 


FIGURE  4-9  \Con' 


FIGURE  4-9  (Continued):  Degree  4  Case. 


FIGURE  4-9  (Continued):  Degree  4  Case- 
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function,  by  a  dotted  line,  for  each  of  the  ten,  unknown  binary  test 
items,  together  with  the  result  obtained  upon  the  original  Old 
Test,  the  theoretical  item  characteristic  function,  and  the 
frequency  ratios  of  the  correct  answer  for  the  subintervals  of 
0  with  the  width  0.25  ,  which  are  drawn  by  a  dashed  line  and 
smooth  and  jagged  solid  lines,  respectively,  for  Degree  3  and  4 
Cases.  Comparison  of  these  two  figures  indicates  that  these  two 
sets  of  results,  i.e.,  those  of  Degree  3  and  4  Cases,  are  practically 
Identical,  the  fact  that  we  observed  in  all  the  previous  studies 
(Samejima,  RR-78-1,  RR-78-2,  RR-78-4,  RR-78-5) .  It  is  also  observed 
that  these  two  curves  for  each  item  are  very  close  to  the  theoretical 
item  characteristic  function,  at  least,  for  the  interval  of  0  , 
(-2.0,  2.0)  .  This  means  that  the  present  method  turned  out  to 
be  successful,  in  spite  of  the  fact  that  the  amount  of  test 
information  is  considerably  small,  especially  for  the  extreme  ranges 
of  ability  0  .  We  also  notice  that  these  estimated  item 
characteristic  functions  are  very  close  to  the  corresponding 
results  obtained  upon  the  original  Old  Test.  To  be  more  precise, 
they  are  practically  identical  for  Items  3,  5,  7,  8  and  10  ,  while 
there  are  some  visible  discrepancies  for  items  1,  2,  4,  6  and  9  . 

It  is  important  to  note  that,  in  general,  these  estimated  item 
characteristic  functions,  which  are  based  upon  Subtest  3,  are  no 
farther  apart  from  the  theoretical  item  characteristic  functions 
than  those  based  upon  the  original  Old  Test,  at  least,  for  the 
interval  of  0  ,  (-2.0,  2.0)  .  This  implies  a  remarkable 
accomplishment  of  the  present  method,  considering  the  fact  that 
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Subtest  3  contains  only  fifteen  test  items,  while  the  original 
Old  Test  has  thirty-five  items,  and  Subtests  1  and  2  have 
twenty-five  test  items  each. 

We  notice,  in  Figures  4-8  and  4-9,  tha,  there  are  some 
items  whose  estimated  item  characteristic  functions  have  lower 
asymptotes  greater  than  zero,  and  also  some  whose  estimated  item 
characteristic  functions  have  upper  asymptotes  less  than  unity. 
Although  the  ranges  of  8  for  which  these  phenomena  are  observed 
are  outside  of  the  meaningful  interval,  (-2.5,  2.5)  ,  it  may  be 
worth  investigating  them.  The  items  which  belong  to  the  first 
group  are  items  1,  2,  3  and  4,  and  those  which  belong  to  the  second 
group  are  items  6,  8,  9  and  10  . 

Table  4-4  presents  the  response  pattern  of  the  ten  unknown, 
binary  test  items  obtained  by  each  of  the  fourteen  hypothetical 
examinees  whose  response  patterns  of  the  fifteen  test  items  of 
Subtest  3  are  uniformly  V-min  ,  or  the  set  of  all  zeros.  We  can 
see  in  this  table  that  for  items  1,  2,  3  and  4  not  all  the  responses 
by  the  fourteen  examinees  are  zero,  i.e.,  eight,  four,  one  and  two 
examinees  out  of  the  fourteen  answered  these  four  items  correctly. 

We  note  from  (4.10)  that  the  ratios  of  these  numbers  to  fourteen 
must  be  the  lower  asymptotes  for  these  four  items,  since  they  are 
the  group  of  examinees  whose  modified  maximum  likelihood  estimates 
are  -2.843)  »  i.e.,  the  lowest.  These  ratios  are 

0.571  ,  0.286  ,  0.071  and  0.143  for  items  1,  2,  3  and  4  , 
respectively,  and  both  Figures  4-8  and  4-9  indicate  that,  indeed, 


TABLE  A -A 


Identification  Number  and  the  Response  Pattern 
of  the  Ten  Unknown,  Binary  Items  Obtained  by 
Each  of  the  Fourteen  Hypothetical  Examinees 
Whose  Response  Patterns  of  Subtest  3  are 
V-min  . 


ID 

Response 

Pattern 

1 

0001000000 

101 

0100000000 

201 

0100000000 

A01 

1000000000 

2 

0100000000 

102 

OOOOOOOCoO 

202 

0000000000 

302 

1000000000 

303 

1000000000 

A 

1100000000 

108 

1000000000 

109 

1001000000 

210 

1000000000 

118 

1010000000 

J - — - 
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they  are  the  lower  asymptotes  for  these  four  estimated  item 
characterist  c  functions  in  both  Degree  3  and  4  Cases.  Similarly, 
Table  4-5  presents  the  response  pattern  of  the  ten  unknown  binary 
test  items  obtained  by  each  of  the  twelve  hypothetical  examinees 
whose  response  patterns  of  the  fifteen  test  items  of  Subtest  3 
are  uniformly  V-max  ,  or  the  set  of  all  2’s  .  This  table  shows 
that  for  items  6,  8,  9  and  10  some  responses  are  zero,  i.e.,  one 
out  of  the  twelve  examinees  answered  items  6,  8  and  9  incorrectly 
and  five  out  of  the  twelve  did  the  same  to  item  10  .  The  ratios 
of  those  who  answered  items  6,  8,  9  and  10  correctly  to  the  total 
number,  twelve,  are  0.583  ,  0.916  ,  0.916  and  0.916  ,  respectively, 
and  they  are  the  upper  asymptotes  of  the  estimated  item 
characteristic  functions  of  the  four  binary  test  items  in  both 
Degree  3  and  A  Cases . 

A  close  examination  of  Tables  4-4  and  4-5  reveals  that  many 
of  the  "unusual"  responses  come  from  the  examinees  whose  true 
ability  levels  are  not  very  low,  or  not  very  high.  To  be  more 
specific,  nine  out  of  the  fifteen  l's  in  Table  4-4  belong  to 
the  six  hypothetical  examinees  whose  true  ability  levels  are 
-2.375  or  higher,  and  seven  out  of  the  eight  0's  in  Table  4-5 
belong  to  the  four  hypothetical  examinees  whose  true  ability  levels 
are  2.375  or  lower.  This  fact  indicates  that  the  small  amounts  of 
test  information  provided  by  Subtest  3  for  these  ranges  of  ability 
0  are  responsible  tor  these  asymptotes,  since  they  are  the  causes 
of  misclassifying  those  examinees  to  V-min  and  V-max  and 


TABLE  4-5 


Identification  Number  and  the  Response  Pattern 
of  the  Ten  Unknown,  Binary  Items  Obtained  by 
Each  of  the  Twelve  Hypothetical  Examinees 
Whose  Response  Patterns  of  Subtest  3  are 
V-max  . 


ID 

Response 

Pattern 

491 

1111111000 

193 

1111111110 

493 

1111111110 

294 

1111111111 

296 

1111111111 

397 

1111111111 

98 

1111111111 

198 

1111101110 

199 

1111111111 

299 

1111111110 

499 

1111111111 

300 

1111111111 
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giving  them  the  lowest  and  the  highest  estimates,  i.e., 
end  T*v_mJu{  ,  respectively. 


SSiT'nrniiir -iri  i  r 
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V  Discussion  and  Conclusions 

The  main  difference  between  the  present  study  and  the  previous 
one  (Samejima,  RR-80-4)  in  which  Subtests  1  and  2  were  used, 
separately,  as  the  Old  Test  lies  in  the  fact  that  the  amount  of 
test  information  provided  by  Subtest  3  is  so  small  at  both  the  lower 
and  higher  extreme  ranges  of  ability  6  ,  that  the  maximum  likelihood 
estimates  of  some  of  the  hypothetical  examinees  are  either  negative 
or  positive  infinity,  and  we  used  the  modified  maximum  likelihood 
estimate  instead,  while  the  same  is  not  the  case  with  either 
Subtest  1  or  Sub test  2.  In  spite  of  this  handicap,  the  results 
of  the  present  study  turned  out  to  be  quite  successful. 

There  is  an  implicit  warning  in  our  results,  however.  As 
was  observed  in  the  preceding  chapter,  these  small  amounts  of  test 
information  provided  by  Subtest  3  for  extreme  values  of  ability 
have  caused  undesirable  asymptotes  for  some  estimated  item 
characteristic  functions.  Although  it  is  relatively  insignificant 
in  the  present  result,  encouragement  in  adopting  a  test  with  small 
amounts  of  information  as  the  Old  Test  will  lead  to  greater 
deviations  of  the  estimated  operating  characteristics  from  the 
theoretical  ones. 

Even  if  the  modified  maximum  likelihood  estimate,  t*  , 
has  an  approximately  linear  regression  on  t  ,  the  deviation  of 
its  conditional  distribution,  given  t  ,  from  the  normality  with 
C  ^  as  its  second  parameter  is  substantial,  as  we  have  observed 
in  Chapter  3.  We  should  not  be  overjoyed,  therefore,  by  the  success 
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In  Che  present  study,  and  become  Insensitive  to  the  shape  of  the 
square  root  of  the  test  information  function  of  a  test,  which 
we  consider  for  the  Old  Test. 

Throughout  the  two  studies,  in  which  we  used  three  tests 
with  non-constant  test  information  functions,  separately,  as  the 
Old  Test,  the  introduction  of  the  transformed  latent  trait  x  proved 
to  be  successful.  The  logical  step  we  should  take  next  will  be 
the  investigation  concerning  the  reduction  of  the  number  of  test 
items  in  our  Old  Test,  which  may  or  may  not  have  a  constant  amount 
of  test  information  for  the  range  of  ability  of  our  interest. 

This  will  be.  done  in  the  near  future,  with  the  warning  pointed 
out  in  the  preceding  paragraph  in  mind. 
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Bolling  Air  Force  Base 
Washington,  DC  20332 

1  Dr.  Earl  A.  Alluisi 
HQ,  AFHRL  (AFSC) 

Brooks  AFB,  TX  78235 

1  Research  and  Measurment  Division 
Research  Branch,  AFMPC/MPCYPR 
Randolph  AFB,  TX  78148 

1  Dr.  Malcolm  Ree 
AFHRL/MP 

Brooks  AFB,  TX  78235 

1  Dr.  Marty  Rockway 
Technical  Director 
AFHRL (0T) 

Williams  AFB,  AZ  58224 

1  Dr.  Frank  Schufletowski 
U.S.  Air  Force 
ATC/XPTD 

Randolph  AFB,  TX  78148 
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Mar  lnes 


H.  William  Greenup 
Education  Advisor  (E031) 

Education  Center,  MCDEC 
Quantico,  VA  2213^ 

Major  Howard  Langdon 
Headquarters,  Marine  Corps 
OTTI  31 

Arlington  Annex 

Colunbia  Pike  at  Arlington  Ridge  Rd. 
Arlington,  VA  20380 

Director,  Office  of  Manpower  Utilization 
HQ,  Marine  Corps  (MPU) 

BCB,  Bldg.  2009 
Quantico.  VA  22134 

Headquarters,  U.  S.  Ma»  Corps 
Code  MPI-20 
Washington,  DC  20380 

Special  Assistant  for  Marine 
Corps  Matters 
Code  100M 

Office  of  Naval  Research 
800  N.  Quincy  St. 

Arlington,  VA  22217 


CoastGuard 


1  C’iief,  Psychological  Reserch  Branch 
U.  S.  Coast  Guard  (G-P-1/2/TP42) 
Washington,  DC  20593 

1  Mr.  Thomas  A.  Warm 

U.  S.  Coast  Guard  Institute 
P.  0.  Substation  18 
Oklahoma  City,  OK  73169 


Major  Michael  L.  Patrow,  USMC 
Headquarters,  Marine  Corps 
(Code  MPI-20) 

Washington,  DC  20380 

DR.  A.L.  SLAFK0SKY 
SCIENTIFIC  ADVISOR  (CODE  RD-1 ) 
HQ,  U.S.  MARINE  CORPS 
WASHINGTON,  DC  20380 
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Other  DoD 


12  Defense  Technical  Information  Center 
Cameron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 

1  Dr.  William  Graham 
Testing  Directorate 
MEPCOM/MEPCT-P 
Ft.  Sheridan,  IL  60037 

1  Director,  Research  and  Data 
OASD(MRAAL) 

3B919,  The  Pentagon 
Washington,  DC  20301 

1  Military  Assistant  for  Training  and 
Personnel  Technology 

Office  of  the  Under  Secretary  of  Defense 
for  Research  &  Engineering 
Room  3D 129.  The  Pentagon 
Washington,  DC  20301 

1  Dr.  Wayne  Sellman 

Office  of  the  Assistant  Secretary 
of  Defense  (MRA  &  L) 

2B269  The  Pentagon 
Washington,  DC  20301 

1  DARPA 

1400  Wilson  Blvd. 

Arlington,  VA  22209 


Civil  Govt 


1  Dr.  Lorraine  D.  Eyde 
Personnel  R&D  Center 
Office  of  Personnel  Management  of  USA 
1900  EStreet  NW 
Washington,  D.C.  20415 

1  Jerry  Lehnus 

REGIONAL  PSYCHOLOGIST 

U.S.  Office  of  Personnel  Management 

230  S.  DEARBORN  STREET 

CHICAGO,  IL  60604 

1  Dr.  Andrew  R.  Molnar 
Science  Education  Dev. 
and  Research 

National  Science  Foundation 
Washington,  DC  20550 

1  Dr.  H.  Wallace  Sinaiko 
Progran  Director 

Manpower  Research  and  Advisory  Services 
Smithsonian  Institution 
801  North  Pitt  Street 
Alexandria,  VA  22314 

1  Dr.  Vern  W.  Urry 

Personnel  R&D  Center 
Office  of  Personnel  Management 
1900  E  Street  NW 
Washington,  DC  20415 

1  Dr.  Joseph  L.  Young,  Director 
Memory  &  Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 
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No  n  Govt 


Dr,  Erling  B.  Andersen 
Department  of  Statistics 
Studiestraede  6 
1455  Copenhagen 
DENMARK 

1  psychological  research  unit 
Dept,  of  Defense  (Army  Office) 
Campbell  Park  Offices 
Canberra  ACT  2600,  Australia 

Dr.  Isaac  Be  jar 
Educational  Testing  Service 
Princeton,  NJ  08450 

Capt.  J.  Jean  Belanger 
Training  Development  Division 
Canadian  Forces  Training  System 
CFTSHQ,  CFB  Trenton 
Astra,  Ontario  KOK  1B0 

Dr .  John  Bergan 
School  of  Education 
University  of  Arizona 
Tuscon  AZ  85721 

CDR  Robert  J.  Biersner 
ogram  Manager 
l.jman  Performance 
Navy  Medical  R&D  Command 
t,ethesda,  MD  20014 

Dr.  Menucha  Birenbaum 

School  of  Education 

Tel  Aviv  University 

Tel  Aviv,  Ramat  Aviv  69978 

Israel 

Dr.  Werner  Birke 
DezWPs  im  Streitkraefteamt 
Postfach  20  50  03 
D-530L  Bonn  2 
WEST  GERMANY 

Dr.  R.  Darrel  Bock 
Department  of  Education 
University  of  Chicago 
Chicago,  IL  60637 


N^n  Govt 


1  Liaison  Scientists 

Office  of  Naval  Research, 

Branch  Office  ,  London 
Box  39  FP0  New  York  09510 

1  Col  Ray  Bowles 
800  N.  Quincy  St. 

Room  804 

Arlington,  VA  22217 

1  Dr.  Robert  Brennan 

American  College  Testing  Programs 

P.  0.  Box  168 

Iowa  City,  IA  52240 

1  DR.  C.  VICTOR  BUNDERSON 
WICAT  INC. 

UNIVERSITY  PLAZA,  SUITE  10 
1160  SO.  STATE  ST. 

OREM,  UT  84057 

1  Dr.  Anthony  Cancel li 
School  of  Education 
University  of  Arizona 
Tuscon,  AZ  85721 

1  Dr.  John  B.  Carroll 
Psychometric  Lab 
Univ.  of  No.  Carolina 
Davie  Hall  01 3A 
Chapel  Hill,  NC  27514 

1  Charles  Myers  Library 
Livingstone  House 
Livingstone  Road 
Stratford 
London  E15  2LJ 
ENGLAND 

1  Dr.  Kenneth  E.  Clark 

College  of  Arts  &  Sciences 
University  of  Rochester 
River  Campus  Station 
Rochester,  NY  14627 
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Non  Govt 


1  Dr.  Norman  Cliff 
Dept,  of  Psychology 
Univ.  of  So.  California 
University  Park 
Los  Angeles,  CA  90007 

1  Dr.  William  E.  Coffman 

Director,  Iowa  Testing  Programs 
334  Lindquist  Center 
University  of  Iowa 
Iowa  City,  IA  52242 

1  Dr.  Meredith  P.  Crawford 

American  Psychological  Association 
1200  17th  Street,  N.W. 

Washington,  DC  20036 

1  Director 

Behavioural  Sciences  Division 
Defence  4  C.  il  Institute  of 
Environmental  Medicine 
Post  Office  Box  2000 
Downsview,  Ontario  M3M  3B9 
CANADA 

1  Dr., Fritz  Drasgow 

Yale  School  of  Organization  and  Manageme 
Yale  University 
Box  1A 

New  Haven,  CT  06520 

1  Dr.  Mavin  D.  Dunnette 

Personnel  Decisions  Research  Institute 
2415  Foshay  Tower 
821  Marguette  Avenue 
neapolis ,  MN  55402 

1  Mike  Durmeyer 

Instructional  Program  Development 

Building  90 

NET-PDCD 

Great  Lakes  NTC,  IL  60088 

1  ERIC  Facility-Acquisitions 
4333  Rugby  Avenue 
Bethesda,  MD  20014 


Non  Govt 


1  Dr.  Benjamin  A.  Fairbank,  Jr. 
McFann-Gray  4  Associates,  Inc. 

5825  Callaghan 
Suite  225 

San  Antonio,  Texas  78228 

1  Dr.  Leonard  Feldt 

Lindquist  Center  for  Measurment 
University  of  Iowa 
Iowa  City,  IA  52242 

1  Dr.  Richard  L.  Ferguson 

The  American  College  Testing  Program 

P.0.  Box  168 

Iowa  City,  IA  52240 

1  Dr.  Victor  Fields 
Dept,  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

1  Univ.  Prof.  Dr.  Gerhard  Fischer 
Liebiggasse  5/3 
A  1010  Vienna 
AUSTRIA 

1  Professor  Donald  Fitzgerald 
University  of  New  England 
Arraidale,  New  South  Wales  2351 
AUSTRALIA 

1  Dr.  John  R.  Frederiksen 
Bolt  Beranek  4  Newman 
50  Moulton  Street 
Cambridge,  MA  02138 

1  DR.  ROBERT  GLASER 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
3939  O’HARA  STREET 
PITTSBURGH,  PA  15213 

1  Dr  .  Ron  Hambleton 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01002 
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Non  Govt 


Dr.  Chester  Harris 
School  of  Education 
University  of  California 
Santa  Barbara,  CA  93106 

Dr .  Lloyd  Humphreys 
Department  of  Psychology 
University  of  Illinois 
Champaign,  IL  61820 

Library 

Hum RRO/ Western  Division 
27857  Berwick  Drive 
Carmel,  CA  93921 

Dr .  Steven  Hunka 
Department  of  Education 
University  of  Alberts 
Edmonton,  Alberta 
CANADA 

Dr.  Jack  Hunter 
2122  Coolidge  St. 

Lansing,  MI  48906 

Dr.  Huynh  Huynh 
College  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

Professor  John  A.  Keats 
University  of  Newcastle 
AUSTRALIA  2308 

Dr.  Michael  Levine 

Department  of  Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  Charles  Lewis 
Faculteit  Sociale  Wetenschappen 
Ri jksuniversiteit  Groningen 
Oude  Boteringestraat  23 
9712GC  Groningen 
Netherlands 


Non  Govt 


1  Dr.  Robert  Linn 

College  of  Education 
University  of  Illinois 
Urbana,  IL  61801 

1  Dr.  Frederick  M.  Lord 

Educational  Testing  Service 
Princeton,  NJ  08540 

1  Dr.  James  Lumsden 

Department  of  Psychology 
University  of  Western  Australia 
Nedlands  W.A.  6009 
AUSTRALIA 

1  Dr.  Gary  Marco 

Educational  Testing  Service 
Princeton,  NJ  08450 

1  Dr .  Scott  Maxwell 

Department  of  Psychology 
University  of  Houston 
Houston,  TX  77004 

1  Dr .  Samuel  T.  Mayo 

Loyola  University  of  Chicago 
820  North  Michigan  Avenue 
Chicago,  IL  60611 

1  Professor  Jason  Millman 
Department  of  Education 
Stone  Hall 
Cornell  University 
Ithaca,  NY  14853 

1  Dr.  Melvin  R.  Novick 

356  Lindquist  Center  for  Measurment 
University  of  Iowa 
Iowa  City,  IA  52242 

1  Dr.  Jesse  Orlansky 

Institute  for  Defense  Analyses 
400  Army  Navy  Drive 
Arlington,  VA  22202 

1  Dr.  James  A.  Paulson 

Portland  State  University 
P.0.  Box  751 
Portland,  OR  97207 
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Non  Govt 


HR.  LUIGI  PETRULLO 
243 1  N.  EDGEWOOD  STREET 
ARLINGTON.  VA  22207 

DR.  DIANE  M.  RAMSEY-KLEE 
R-K  RESEARCH  4  SYSTEM  DESIGN 
3947  RIDGEMONT  DRIVE 
MALIBU,  CA  90265 

MINRAT  M.  L.  RAUCH 
P  II  4 

BUN DESM IN ISTER IUM  DER  VERTEIDIGUNG 

POSTFACH  1328 

D-53  BONN  1,  GERMANY 

Dr.  Mark  D.  Reckase 
Educational  Psychology  Dept. 
University  of  Missour  i-Colunbia 
4  Hill  Hall 
Columbia ,  MO  65211 

Dr.  Leonard  L.  Rosenbaum,  Chairman 
Department  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

Dr .  Lawrence  Rudner 
403  Elm  Avenue 
Takoma  Park,  MD  20012 

Dr  .  J .  Ryan 

Department  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

DR.  ROBERT  J.  SEIDEL 
INSTRUCTIONAL  TECHNOLOGY  GROUP 
HUMRRO 

300  N.  WASHINGTON  ST. 

ALEXANDRIA,  VA  22314 

Dr.  Kazuo  Shigemasu 

University  of  Tohoku 

Department  of  Educational  Psychology 

Kawauchi,  Sendai  980 

JAPAN 


Non  Govt 


1  Dr.  Edwin  Shirkey 

Department  of  Psychology 
University  of  Central  Florida 
Orlando,  FL  32816 

1  Dr  .  Robert  Smith 

Department  of  Computer  Science 

Rutgers  University 

New  Brunswick,  NJ  08903 

1  Dr.  Richard  Snow 
School  of  Education 
Stanford  University 
Stanford,  CA  94305 

1  DR.  PATRICK  SUPPES 

INSTITUTE  FOR  MATHEMATICAL  •’TUDIES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD,  CA  94305 

1  Dr.  Hariharan  Swaminathan 

Laboratory  of  Psychometric  and 
Evaluation  Research 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01003 

1  Dr.  Brad  Sympson 

Psychometric  Research  Group 
Educational  Testing  Service 
Princeton,  NJ  08541 

1  Dr.  Kikumi  Tatsuoka 

Computer  Based  Education  Research 
Laboratory 

252  Engineering  Research  Laboratory 
University  of  Illinois 
Urbana,  IL  61801 

1  Dr.  David  Thissen 

Department  of  Psychology 
University  of  Kansas 
Lawrence,  KS  66044 

1  Dr.  Robert  Tsutakawa 

Department  of  Statistics 
University  of  Missouri 
Columbia,  M0  65201 
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Non  Govt 


Dr.  J.  Uhlaner 
Perceptronics,  Inc. 

6271  Variel  Avenue 
Woodland  Hills,  CA  91 364 

Dr.  Howard  Wainer 
Division  of  Psychological  Studies 
Educational  Testing  Service 
Princeton,  NJ  08540 

Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455 

DR.  SUSAN  E.  WHITELY 
PSYCHOLOGY  DEPARTMENT 
UNIVERSITY  OF  KANSAS 
LAWRENCE,  KANSAS  66044 

Wolfgang  Wildgrube 
Streitkraefteamt 
Box  20  50  03 
D-5300  Bonn  2 


i 


j 

3 

1 

I 


I 

i 


i 

i 

3 


3 


i 

;  i 


-130- 


Navy 


1  Ms.  Nancy  McHan 

Office  of  Naval  Research 
206  O'Keefe  Building 
Atlanta,  GA  30332 


Army 

1  Dr.  Randall  M.  Chambers 

U.S.  Army  Research  Institute 
for  the  Behavioral  &  Social 
Sciences 

Fort  Sill  Field  Unit 
P.0.  Box  3066 
Fort  Sill,  OK  73503 


Non  Govt 


1  Dr.  Bert  F.  Green 

Department  of  Psychology 
The  John’s  Hopkins  University 
Charles  at  34th  Street 
Baltimore,  MD  21218 

1  Dr.  Ron  Hambleton 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01002 

1  Dr.  William  W.  Turnbull 

Educational  Testing  Service 
Princeton,  NJ  08540 

1  Dr.  Isaac  I.  Be jar 

Department  of  Psychology 
Elliot  Hall 
75  East  River  Road 
Minneapolis,  MN  55455 

1  Dr.  George  Woods 

1106  Newport  Ave. 

Victoria,  B.  C. 

V8S  5E4  Canada 

1  Dr.  Lowell  Schipper 

Department  of  Psychology 
Bowling  Green  State  University 
Bowling  Green,  OH  43403 


Uon  Govt 


1  Dr.  P.  Mengal 

Faculte*  de  Psychologie 
et  des  Sciences  de  l'Educatlon 
Universite'  de  Geneve 
3  fl.  de  l'Universite 
1201  Geneva  SWITZERLAND 

1  Dr.  Wim  J.  van  der  Linden 

Vakgroep  Onderwisjskunde 
Postbus  217 
7500  EA  Enschede 
The  Netherlands 

1  Dr.  Lutz  Hornke 

University  Duesseldorf 
Erz.  Wiss. 

D-4000  Duesseldorf 
WEST  GERMANY 

1  Dr.  Wolfgang  Buchtala 

8346  Simbach  Inn 
Postfach  1306 
Industriestrasse  1 
WEST  GERMANY 


1  Dr.  Sukeyori  Shiba 

Faculty  of  Education 
University  of  Tokyo 
Hongo ,  Bumkyoku 
Tokyo,  Japan  113 

1  Mr.  Yukihiro  Noguchi 

Faculty  of  Education 
University  of  Tokyo 
Hongo ,  Bumkyoku 
Tokyo,  Japan  113 

1  Dr.  Takahiro  Sato  (Representative) 
Application  Research  Laboratory 
Central  Research  Laboratories 
Nippon  Electric  Co.,  Ltd. 

4-1-1  Miyazaki,  Takatsu-ku 
Kawasaki  213,  Japan 
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Non  Govt 


1  Dr.  James  Chen 

334  Lindquist  Center  for  Measurement 
University  of  Iowa 
lows  City,  lows  52240 

1  Mr.  Jeffrey  Jsnkowits 

Department  of  Educational  Psychology 
210  Education  Building 
University  of  Illinois 
Urbane,  IL  61801 

1  Dr.  Douglas  Carroll 
Bell  Laboratories 
600  Mountain  Ave. 

Murray  Hill,  N.J.  07974 

1  Dr.  Robert  Guion 

Department  of  Psychology 
Bowling  Green  State  University 
Bowling  Green,  OH  43403 

1  Dr.  Niel  Timm 

Department  of  Information  Systems 
Planning 

University  of  Pittsburgh 
Pittsburgh,  PA  15260 

1  Dr.  Albert  Beaton 

Educational  Testing  Service 
Princeton,  New  Jersey  08450 


